A variational Bayes approach to variable selection

Empirical Bayes vs. Fully Bayes Variable Selection

For the problem of variable selection for the normal linear model, fixed penalty selection criteria such as AIC, Cp, BIC and RIC correspond to the posterior modes of a hierarchical Bayes model for various fixed hyperparameter settings. Adaptive selection criteria obtained by empirical Bayes estimation of the hyperparameters have been shown by George and Foster [2000. Calibration and Empirical B...

A Variational Bayes Approach to Robust Principal Component Analysis

We solve the Robust Principal Component Analysis problem: decomposing an observed matrix into a low-rank matrix plus a sparse matrix. Unlike alternative methods that approximate this l0 objective with an l1 objective and solve a convex optimization problem, we develop a corresponding generative model and solve a statistical inference problem. The main advantages of this approach is its ability ...

A Hierarchical Bayes Approach to Variable Selection for Generalized Linear Models

For the problem of variable selection in generalized linear models, we develop various adaptive Bayesian criteria. Using a hierarchical mixture setup for model uncertainty, combined with an integrated Laplace approximation, we derive Empirical Bayes and Fully Bayes criteria that can be computed easily and quickly. The performance of these criteria is assessed via simulation and compared to othe...

Calibration and empirical Bayes variable selection

For the problem of variable selection for the normal linear model, selection criteria such as , C p ,  and  have fixed dimensionality penalties. Such criteria are shown to correspond to selection of maximum posterior models under implicit hyperparameter choices for a particular hierarchical Bayes formulation. Based on this calibration, we propose empirical Bayes selection criteria that...

The Variational Bayes Approach in Signal Processing